Adaptive buffering in a distributed system with latency / adaptive tail drop

ABSTRACT

A network device includes a switching system for directing packets between ingress ports and egress ports of the network device. The network device also includes a switching system manager that makes an identification of a state change of a virtual output queue of the switching system; and performs an action set, based on the state change, to modify a latency of the virtual output queue to meet a predetermined latency in response to the identification.

BACKGROUND

Computing devices may utilize computing resources to perform theirfunctionality. For example, computing devices may utilize processingresources to execute instructions, memory resources to temporarily storedata, storage resources to store data long term, etc. The quantity ofeach of these resources may be limited.

SUMMARY

In one aspect, a network device in accordance with embodiments disclosedherein includes a switching system for directing packets between ingressports and egress ports of the network device; and a switching systemmanager that makes an identification of a state change of a virtualoutput queue of the switching system; and performs an action set, basedon the state change, to modify a latency of the virtual output queue tomeet a predetermined latency in response to the identification.

In one aspect, a method in accordance with one or more embodimentsdisclosed herein includes making an identification of a state change ofa virtual output queue of a switching system for directing packetsbetween ingress ports and egress ports of a network device; performingan action set, based on the state change, to modify a latency of thevirtual output queue to meet a predetermined latency in response to theidentification.

In one aspect, a non-transitory computer readable medium in accordancewith one or more embodiments disclosed herein includes computer readableprogram code, which when executed by a computer processor enables thecomputer processor to perform a method, the method includes making anidentification of a state change of a virtual output queue of aswitching system for directing packets between ingress ports and egressports of a network device; performing an action set, based on the statechange, to modify a latency of the virtual output queue to meet apredetermined latency in response to the identification.

BRIEF DESCRIPTION OF DRAWINGS

Certain embodiments disclosed herein will be described with reference tothe accompanying drawings. However, the accompanying drawings illustrateonly certain aspects or implementations of the embodiments disclosedherein by way of example and are not meant to limit the scope of theclaims.

FIG. 1.1 shows a diagram of a system in accordance with one or moreembodiments disclosed herein.

FIG. 1.2 shows a diagram of a network device in accordance with one ormore embodiments disclosed herein.

FIG. 1.3 shows a diagram of a switching system in accordance with one ormore embodiments disclosed herein.

FIG. 1.4 shows a diagram of a switching system repository in accordancewith one or more embodiments disclosed herein.

FIG. 1.5 shows a diagram of switching system operational parameters inaccordance with one or more embodiments disclosed herein.

FIG. 1.6 shows a relationship diagram of relationships in accordancewith one or more embodiments disclosed herein.

FIG. 2.1 shows a flowchart of managing a switching system in accordancewith one or more embodiments disclosed herein.

FIG. 2.2 shows a flowchart of method of managing latency states inaccordance with one or more embodiments disclosed herein.

FIGS. 3.1-3.6 show a non-limiting example of a system at differentpoints in time and/or relationships used by the system in accordancewith embodiments disclosed herein.

FIG. 4 shows a diagram of a computing device in accordance with one ormore embodiments disclosed herein.

DETAILED DESCRIPTION

Networks may include devices that generate, send, receive, and/orforward packets. A packet may be forwarded by obtaining the packet on aningress port, switching the packet to an egress port, and providing thepacket to another device via the egress port.

A network device that forwards packets may have a limited ability toforward packets based on the architecture it uses to switch the packets.For example, the network device may only be able to forward apredetermined number of packets per using each of their egress ports(e.g., 100 Gigabits per second) per unit time. In another example, thenetwork device may only be able to buffer a limited number of packets.

Additionally, the network device's architecture for forwarding packetsmay be configurable. For example, the network device may include afinite amount of resources usable for buffering that may be allocatedacross a number of ports. The forwarding behavior of the network devicemay be undesirable if the resources usable for buffering are allocated,for example, proportionally across all of the ports based on therespective rates at which packets enter each of the ports. Allocatingthe resources in this manner may have undesirable consequences fornetwork wide routing such as increased latency of packets. The increasedlatency of the packets may result in other components of the network (i)resending copies of the packets and/or (ii) discarding the packets afterthe packets are buffered.

One or more embodiments disclosed herein provide methods and systems formanaging the latency of packets that are buffered during forwarding ofthe packets. To do so, a system in accordance with embodiments disclosedherein may monitor the latency states of virtual output queues used tobuffer packets during forwarding. The system may allocate resources(e.g., reduce or increase the number of packets that may be buffered ineach of the virtual output queues) to the virtual output queues based onthe latency states of the virtual output queues to reduce the likelihoodthat packets are buffered for greater than a predetermined amount oftime. In other words, to meet latency goals for packets duringforwarding.

The system may also proactively drop packets that would otherwise bebuffered, based on the latency states of the virtual output queues.Doing so may reduce the likelihood that packets are buffered for greaterthan a predetermined amount of time (e.g., a latency goal). Packets maybe proactively dropped even when the virtual output queues havesufficient capacity (e.g., storage resources) to buffer additionalpackets.

By proactively dropping packets based on the latency states of thevirtual output queues, embodiments disclosed herein may (i) avoiddeleterious downstream impacts on packet routing, (ii) indirectly signalto upstream entities (by virtue of dropped packets) that the networkdevice is unable to forward the current volume of packets which it isobtaining for forwarding purposes, and/or (iii) avoid consumption ofvirtual output queue storage space for packets that will be discardeddownstream due to the latency of the packets caused by buffering in thevirtual output queue.

Thus, embodiments disclosed herein may provide a network device thatmore efficiently marshals (i) its own limited resources by avoidingbuffering that does not positively contribute to packet routing and/or(ii) resources of other devices by proactively dropping packets thatwill likely become high latency packets after buffering. Consequently,downstream devices may be prevented from using their resources forprocessing the high latency packets that will likely be discarded.

In the text that follows, a description of components of a system inaccordance with embodiments disclosed herein is provided with respect toFIGS. 1.1-1.3 . A description of data structures that may be used by thesystem of FIG. 1.1 is provided with respect to FIGS. 1.4-1.5 . After thedescription of the data structures, a description of relationships thatmay be present in the system of FIG. 1.1 is provided with respect toFIG. 1.6 .

Following the description of the relationships, a description of methodsthat may be performed by components of the system of FIG. 1.1 isprovided with respect to FIGS. 2.1-2.2 . An example of a series ofactions that may be performed by the system of FIG. 1.1 is provided withrespect to FIGS. 3.1-3.6 . Lastly, a description of a computing devicethat may be used to implement the system of FIG. 1.1 is provided withrespect to FIG. 4 .

Specific embodiments will now be described with reference to theaccompanying figures. In the following description, numerous details areset forth as examples. It will be understood by those skilled in theart, and having the benefit of this document, that one or moreembodiments described herein may be practiced without these specificdetails and that numerous variations or modifications may be possiblewithout departing from the scope of the embodiments. Certain detailsknown to those of ordinary skill in the art may be omitted to avoidobscuring the description.

In the following description of the figures, any component describedwith regard to a figure, in various embodiments, may be equivalent toone or more like-named components shown and/or described with regard toany other figure. For brevity, descriptions of these components may notbe repeated with regard to each figure. Thus, each and every embodimentof the components of each figure is incorporated by reference andassumed to be optionally present within every other figure having one ormore like-named components. Additionally, in accordance with variousembodiments described herein, any description of the components of afigure is to be interpreted as an optional embodiment, which may beimplemented in addition to, in conjunction with, or in place of theembodiments described with regard to a corresponding like-namedcomponent in any other figure.

Throughout the application, ordinal numbers (e.g., first, second, third,etc.) may be used as an adjective for an element (i.e., any noun in theapplication). The use of ordinal numbers is not to imply or create anyparticular ordering of the elements nor to limit any element to beingonly a single element unless expressly disclosed, such as by the use ofthe terms “before”, “after”, “single”, and other such terminology.Rather, the use of ordinal numbers is to distinguish between theelements. By way of an example, a first element is distinct from asecond element, and the first element may encompass more than oneelement and succeed (or precede) the second element in an ordering ofelements.

As used herein, the phrase operatively connected, operably connected, oroperative connection, means that there exists betweenelements/components/devices a direct or indirect connection that allowsthe elements to interact with one another in some way. For example, thephrase ‘operably connected’ may refer to any direct (e.g., wireddirectly between two devices or components) or indirect (e.g., wiredand/or wireless connections between any number of devices or componentsconnecting the operatively connected devices) connection. Thus, any paththrough which information may travel may be considered an operableconnection.

FIG. 1.1 shows a diagram of an example system in accordance with one ormore embodiments described herein. The system may include clients (5)that utilize services provided by network device (10). The servicesprovided by the network device (10) may include, for example, packetforwarding services. By forwarding packets, network device (10) mayenable clients (5) to communicate with other devices (e.g., otherclients, network (20), or other network devices (30)).

For example, network device (10) may be operably connected to othernetwork devices (30) via network (20). Network device (10), network(20), and other network devices (30) may cooperate (e.g., all implementone or more common network communication protocols) to forward packetsto each other. Network device (10) may also forward packets to clients(5). The packets may include messages destined for the various devicesof the example system.

Forwarding packets may consume computing resources of the devices ofFIG. 1.1 . For example, when a packet is sent from clients (5) tonetwork device (10), network device (10) may need to perform one or moreactions to determine to which device (e.g., a device of network (20) oranother device not illustrated in FIG. 1.1 ) to forward the packet. Toforward a packet, network device (10) may receive the packet on aningress port and send the packet out of an egress port that directs thepacket towards a destination. Network device (10) may include multipleingress ports and egress ports. Consequently, network device (10) mayneed to selectively direct (e.g., switch) packets towards correspondingegress ports to direct the packets towards corresponding destinations.

Additionally, any of the aforementioned ports may be bi-directional andmay dynamically change between being ingress and an egress port at anytime. Thus, any port may perform the functions of an ingress port,egress port, or both at any point in time.

While each of the devices of FIG. 1.1 are illustrated as being operablyconnected to other devices using lines having double ended arrows asends, such lines may represent any number of physical connections overwhich the packets may be forwarded. For example, the line betweennetwork device (10) and network (20) may represent ten physicalconnections between network device (10) and devices of network (20). Insuch a scenario, packets received by network device (10) may beforwarded towards network (20) and/or other network devices (30) usingany of the ten physical connections.

In another example, the line between network device (10) and network(20) may represent a connection between network device (10) and anotherdevice (not shown) which is, in turn, physically connected to a deviceof network (20). In such a scenario, packets received by network device(10) may be forwarded towards network (20) and/or other network devices(30) using the connection to the other device (not shown) and theconnection between the other device and the device of network (20).

Each of the aforementioned physical connections between network device(10) and other devices may have limited bandwidths. In other words, onlya finite quantity of packets per unit time (e.g., packet traffic) may betransmitted between network device (10) and other devices via each ofthe physical connections interconnecting the aforementioned devices.Consequently, if the number of packets received by network device (10)and that need to be sent via a particular physical connection towardsanother device exceeds the finite number of data units per unit timethat may be transmitted via the particular physical connection, networkdevice (10) may need to buffer some of the packets for sending in thefuture.

However, buffering packets may incur latency (e.g., delay time). If thelatency exceeds predetermined amounts, the latency may have a negativeimpact on the ability of the system to forward packets. For example,communication protocols may cause packets having latencies that exceed athreshold to be dropped. In such a scenario, additional packets (e.g.,replacement packets) may need to be forwarded throughout the system toreplace the dropped packets.

Embodiments disclosed herein may provide a network device (e.g., 10)that manages the manner in which packets are processed for forwardingpurposes to provide a predetermined quality of forwarding service. Thepredetermined quality of forwarding service may limit the maximum amountof time that the packets are buffered. In other words, the maximumamount of latency during buffering for packet forwarding purposes may belimited to a predetermined amount of time. For additional detailsregarding limiting the latency for buffering by network device (10) forforwarding purposes, refer to FIG. 1.2 .

Any of the components of FIG. 1.1 may be operably connected by anycombination and/or number of wired and/or wireless connections.

As discussed above, the system of FIG. 1.1 may include network devicesthat may provide packet forwarding services. Any of the devices of FIG.1.1 may be implemented using computing devices. The computing devicesmay be, for example, mobile phones, tablet computers, laptop computers,desktop computers, servers, switches, or cloud resources. The computingdevices may include one or more processors, memory (e.g., random accessmemory), and persistent storage (e.g., disk drives, solid state drives,etc.). The persistent storage may store computer instructions, e.g.,computer code, that (when executed by the processor(s) of the computingdevice) cause the computing device to perform the functions described inthis application and/or all, or a portion, of the methods illustrated inFIGS. 2.1-2.2 . The devices of FIG. 1.1 may be implemented using othertypes of computing devices without departing from the embodimentsdisclosed herein. For additional details regarding computing devices,refer to FIG. 4 .

The devices of FIG. 1.1 may be implemented using logical devices withoutdeparting from the embodiments disclosed herein. For example, thedevices of FIG. 1.1 may be implemented using virtual machines thatutilize computing resources of any number of physical computing devicesto provide their respective functionalities. The devices of FIG. 1.1 maybe implemented using other types of logical devices without departingfrom the embodiments disclosed herein.

In one or more embodiments disclosed herein, the network devices areimplemented as a switching device such as a switch, multilevel switch,or another type of devices that provides packet switching services. Aswitching device may be a device that is adapted to facilitate networkcommunications. A switching device may include a computing device and/orother components/devices. Refer to FIG. 4 for additional detailsregarding a computing device.

While the system of FIG. 1.1 has been illustrated as including a limitednumber of specific components, a system in accordance with embodimentsdisclosed herein may include additional, fewer, and/or differentcomponents.

To further clarify aspects of network devices, a diagram of networkdevice (10) is provided in FIG. 1.2 . Any of the network devices of FIG.1.1 may be similar to network device (10) illustrated in FIG. 1.2 .

FIG. 1.2 shows a diagram of network device (10) in accordance with oneor more embodiments described herein. Network device (10) may facilitatenetwork communications. To do so, network device (10) may provide anynumber of functionalities, including packet forwarding functionality.

To provide packet forwarding functionality, network device (10) mayinclude ingress ports (102), egress ports (104), switching system (106)that switches packets received via ingress ports (102) to egress ports(104), switching system manager (108) that manages the operation ofswitching system (106), and/or switching system repository (110)maintained by switching system manager (108). Each of these componentsof network device (10) is discussed below.

Ingress ports (102) may facilitate obtaining of packets from otherdevices. For example, each ingress port of ingress ports (102) may beadapted to operably connect to other devices. Ingress ports (102) mayobtain the packets from the other devices via the operable connections.

Egress ports (104) may facilitate providing of network units to otherdevices. For example, each egress port of egress ports (104) may beadapted to operably connect to other devices. Egress ports (104) mayprovide the packets to other devices via the operable connections.

Switching system (106) may forward packets obtained via ingress ports(102) to other devices via egress ports (104). For example, switchingsystem (106) may direct packets received via ingress ports (102) towardsegress ports (104). Switching system (106) may selectively direct (e.g.,switch) the packets in a manner that facilitates controlling the flow ofpackets (e.g., packet traffic) across the network illustrated in FIG.1.1 . For additional details regarding switching system (106), refer toFIG. 1.3 .

Switching system manager (108) may manage the operation of switchingsystem (106). For example, switching system manager (108) may manage (i)the rate at which packets received via ingress ports (102) areprocessed, (ii) allocation of storage resources of switching system(106) for buffering of packets, and/or (iii) other aspects of theoperation of switching system (106).

To manage the operation of switching system (106), switching systemmanager (108) may (i) monitor the operation of switching system, (ii)identify a state of the switching system based on the monitoring, and/or(iii) modify the operation of switching system (106) based on theidentified state of switching system (106). In one or more embodimentsdisclosed herein, the state of switching system (106) may reflect, inpart, the latency of packets processed by switching system (106) (e.g.,a latency state), and/or portions thereof. Switching system manager(108) may modify the operation of switching system (106) based on thelatency states of one or more portions of switching system (106) to meetpredetermined latency goals. For example, the latency goals may be tolimit the latency of packets processed by the portions of switchingsystem (106) to a predetermined amount (e.g., 5 milliseconds or anotherfinite amount of time) or to limit the duration of packet buffering inswitching system (106) to a predetermined amount.

In one or more embodiments disclosed herein, switching system manager(108) is implemented using a hardware device including circuitry.Switching system manager (108) may include, for example, a digitalsignal processor, a field programmable gate array, and/or an applicationspecific integrated circuit. The circuitry of the hardware devices maybe adapted to provide the functionality of switching system manager(108). Switching system manager (108) may be implemented using othertypes of hardware devices without departing from the embodimentsdisclosed herein.

In one or more embodiments disclosed herein, switching system manager(108) is implemented using computing code stored on a persistent storagethat when executed by a processor performs the functionality ofswitching system manager (108). The processor may be a hardwareprocessor including circuitry such as, for example, a central processingunit or a microcontroller. The processor may be other types of hardwaredevices for processing digital information without departing from theembodiments disclosed herein.

Switching system manager (108) may perform all, or a portion, of themethods illustrated in FIGS. 2.1-2.2 as part of providing itsfunctionality.

Switching system repository (110) may store information regarding theoperation of switching system (106). The stored information may specifythe operation of switching system (106) and/or the manner in whichswitching system manager (108) manages the operation of switching system(106).

For example, switching system repository (110) may store informationthat specifies (i) operational characteristics of buffers (e.g., queuesin which packets are temporarily stored) of switching system (106), (ii)rates at which packets stored in buffers of switching system (106) areprocessed, and/or (iii) information regarding the states (e.g., latencystates) of the buffers of switching systems (106). Switching systemmanager (108) may maintain and/or utilize the states of the buffersspecified by switching system repository (110) when managing theoperation of switching system (106).

In one or more embodiments disclosed herein, switching system repository(110) is implemented using devices that provide data storage services(e.g., storing data and providing copies of previously stored data). Thedevices that provide data storage services may include hardware devicesand/or logical devices. For example, switching system repository (110)may include any quantity and/or combination of memory devices (i.e.,volatile storage), long term storage devices (i.e., persistent storage),other types of hardware devices that may provide short term and/or longterm data storage services, and/or logical storage devices (e.g.,virtual persistent storage/volatile storage).

For example, switching system repository (110) may include a memorydevice (e.g., a dual in line memory device) in which data is stored andfrom which copies of previously stored data are provided. In anotherexample, switching system repository (110) may include a persistentstorage device (e.g., a solid state disk drive) in which data is storedand from which copies of previously stored data is provided. In a stillfurther example, switching system repository (110) may include a memorydevice (e.g., a dual in line memory device) in which data is stored andfrom which copies of previously stored data are provided and apersistent storage device that stores a copy of the data stored in thememory device (e.g., to provide a copy of the data in the event thatpower loss or other issues with the memory device that may impact itsability to maintain the copy of the data cause the memory device to losethe data).

Switching system repository (110) may store data structures includingthe information discussed above. For additional details regarding datastructures that may be stored in switching system repository (110),refer to FIGS. 1.4-1.5 .

While network device (10) of FIG. 1.2 has been illustrated as includinga limited number of specific components, a system in accordance withembodiments disclosed herein may include additional, fewer, and/ordifferent components.

As discussed above, network device (10) may include switching system(106). FIG. 1.3 shows a diagram of switching system (106) in accordancewith one or more embodiments disclosed herein. Switching system (106)may facilitate switching of packets between any number of ingress ports(e.g., 102A, 102L, 102N) and any number of egress ports (e.g., 104A,104N).

Switching in accordance with one or more embodiments disclosed hereinmay be a process of directing packets received via an ingress port toone or more egress ports (and to another device via the egress ports).Packets received via an ingress port may be switched towards any numberof egress ports. For example, a first packet obtained via a firstingress port may be switched to a first egress port, a second packetobtained via the first ingress port may be switched to a second egressport, a third packet obtained via the first ingress port may be switchedto the first egress port, etc. Any packet obtained via the first ingressport may be switched to any of the egress ports.

The packets may be switched based on, for example, the destinations ofthe packets (e.g., part of control information of the packets). Thepackets may be switched based on additional and/or different informationwithout departing from embodiments disclosed herein. For example, thepackets may be switched based on a network topology, congestion of anetwork, and/or other factors that impact the flow of packets across anetwork.

To switch packets, switching system (106) may be operably connected toingress ports (e.g., 102A, 102L, 102N) and egress ports (e.g., 104A,104N). Switching system (106) may be operably connected to similarnumbers (e.g., 10 ingress ports and 10 egress ports) or differentnumbers (e.g., 6 ingress ports and 12 egress ports) of ingress andegress ports.

To perform switching, switching system (106) may include any number oftraffic processors (e.g., 112A, 112N) and switching fabric (118). Eachof these components of switching system (106) is discussed below.

Traffic processors (e.g., 112A) may process packets for switchingpurposes. Processing packets for switching purposes may include (i)obtaining packets from one or more ports (e.g., 102A, 102L), (ii)selectively dropping any number (e.g., a portion or all) of the packets,and/or (iii) distributing the packets that are not dropped to virtualoutput queues (116) of the traffic processors that each buffer packetsthat are to be sent out of a corresponding egress port. In other words,all packets that are to be sent out of a first egress port may bebuffered in a first virtual output queue, all packets that are to besent out of a second egress port may be buffered in a second virtualoutput queue, etc. Packets may also be switched to the virtual outputqueues based on other characteristics such as traffic class.

For example, the traffic processor may include a packet processor (114)that selectively drops packets and distributes the packets that are notdropped to the virtual output queues (e.g., 116A, 116N). By doing so,each of the traffic processors (e.g., 112A, 112N) may each include avirtual output queue that stores all packets received by the respectivetraffic processor that are to be sent out of each corresponding egressport.

To manage traffic, traffic processors (e.g., 112A, 112N) may selectivelydrop packets based on characteristics of virtual output queues (116) towhich the packets will be distributed. For example, when a packet is tobe switched from an ingress port to an egress port, the probability thatthe packet will be dropped before queueing (i.e., placed in a virtualoutput queue) may depend on (i) the rate packets are being enqueued inthe virtual output queues, (ii) an amount of storage resources allocatedto the virtual output queues, (iii) a percentage (or other type ofquantification) of storage resources allocated to the virtual outputqueues that are in use, (iv) the rate at which packets are being removed(e.g., dequeued) from the virtual output queues, (v) the latency ofpackets buffered in the virtual output queues, and/or (vi) latencystates of the virtual output queues. Consequently, the selectivedropping of packets for each of the virtual output queues (116) may bechanged as the operational characteristics (e.g., rate packets are beingenqueued to the virtual output queues, storage resources allocated tothe virtual output queues, etc.) of the virtual output queues (116)change over time.

The probability that a packet will be dropped by the packet processorbefore being queued may be stored as part of switching system repository(110, FIG. 1.2 ). For example, traffic processors (e.g., 112A, 112N) mayread (or otherwise use) the aforementioned probability from switchingsystem repository (110, FIG. 1.2 ) and drop packets as specified by theprobabilities. Switching system manager (108, FIG. 1.2 ) may maintainthe aforementioned probabilities in switching system repository (110,FIG. 1.2 ). The aforementioned probabilities may be stored in otherlocations without departing from embodiments disclosed herein.

Packet processors (114) may implement any protocol for enqueuing packetsto virtual output queues (116) without departing from embodimentsdisclosed herein. In one or more embodiments disclosed herein, packetprocessor (114) enqueues packets on virtual output queues mapping to anegress port (and/or at a given traffic class). Virtual output queues arediscussed in more details with respect to switching fabric (118), below.

Virtual output queues (116) may temporarily store (e.g., buffer) packetsuntil processed by switching fabric (118). Virtual output queues (116)may be operated as, for example, first in first out queues.

Each virtual output queue of the traffic processors (e.g., 112A, 112N)may temporarily store packets that will be provided to other devices viacorresponding egress ports. For example, all packets received by atraffic processor that will be provided to other devices via a firstegress port may be stored in a virtual output queue, all packets thatwill be provided to other devices via a second egress port may be storedin a second virtual output queue, etc. Consequently, each trafficprocessor may include a virtual output queue for storing packets thatwill be switched to the same egress port for packet forwarding purposes.

Quantities of storage resources of traffic processor (e.g., 112A) may bedynamically allocated to each virtual output queue of the trafficprocessor (e.g., 112A) for buffering purposes. The quantity of storageresources may be reallocated between virtual output queues (e.g., 116A,116N) of a traffic processor (e.g., 112A) to meet packet processinggoals (e.g., latency goals), as will be discussed in greater detailbelow. Traffic processors (e.g., 112A, 112N) may include any quantity ofstorage resources.

Traffic processors (e.g., 112A, 112N) may be implemented using one ormore physical devices. A physical device (e.g., a chip, die, etc.) mayinclude circuitry adapted to perform the functionality of trafficprocessors (e.g., 112A, 112N). In some embodiments disclosed herein, thecircuitry may include programmable portions that may be adapted toexecute computing instructions (e.g., computer code) that cause theprogrammable portions of the circuitry to perform all, or a portion, thefunctionality of traffic processors (e.g., 112A, 112N).

Virtual output queues (116) may be implemented using any methodologywithout departing from embodiments disclosed herein. For example,virtual output queues (116) may be implemented as in-memory datastructures in which packets are temporarily stored. The packets may bearranged in memory to form queues corresponding to each virtual outputqueue of virtual output queues (116). Virtual output queues (116) may beimplemented using special purpose hardware (e.g., buffers) of trafficprocessor A (112A).

Virtual output queues (116) may include a virtual output queuecorresponding to each egress port to which packets received by trafficprocessors A (112A) may be switched for forwarding purposes. Any of theother traffic processors (e.g., 112N) may include similar or differentnumbers of virtual output queues depending on whether they each forwardpackets to similar or different sets of egress ports (104A, 104N).

Switching system (106) may include any number of traffic processors(e.g., 112A, 112N). Each of the traffic processors may process packetsobtained from any number of ingress ports (e.g., a single ingress port,two ingress ports, etc.). Each of the traffic processors may include anynumber of virtual output queues. Different traffic processors mayinclude similar or different numbers of virtual output queues.

While the traffic processors have been illustrated in FIG. 1.3 asincluding a limited number of specific components, a traffic processorin accordance with embodiments disclosed herein may include additional,fewer, and/or different components than those illustrated in FIG. 1.3 .

Switching fabric (118) may process packets obtained from virtual outputqueues of traffic processors. Processing packets may include switchingpackets from virtual output queues of traffic processors to egress ports(e.g., 104A, 104N). In other words, packets may be processed byproviding packets to other devices via egress ports.

To switch packets, switching fabric (118) may direct packets from avirtual output queue of a traffic processor to an egress port out ofwhich the packets in the virtual output queue are to be sent. Any numberof traffic processors may include a virtual output queue in whichpackets are enqueued that are to be sent out of the same egress port. Inother words, an N to 1 relationship may exist between virtual outputqueues of the traffic processors and the egress port out of whichenqueued packets will be sent. Because the rate at which packets may besent out of any egress port is limited, the rate at which packets areenqueued in these virtual output queues may exceed that rate at whichswitching fabric (118) is able to dequeue and send them out of theegress port.

By utilizing virtual output queues, a one to one relationship between agroup of virtual output queues and an egress port (e.g., for a giventraffic class) may be established (in contrast to a many to onerelationship between virtual output queues that buffer packets that willbe provided to other devices via a single egress port). Theaforementioned relationship may be utilized to efficiently switchpackets from traffic processors to egress ports.

To switch packets from virtual output queues to egress ports, switchingfabric (118) may process packets from virtual output queues in apredetermined order. Depending on the processing order, the relativeweight at which packets are dequeued from each virtual output queue thatenqueues packets to be sent out of an egress port may be modified. Theprocessing order of packets may be adjustable.

In one or more embodiments disclosed herein, switching fabric (118)processes packets buffered in virtual output queues based on weights(e.g., processing rates of assigned to each of the virtual output queuesof a virtual output queue) assigned to each of the virtual outputqueues. The switching fabric (118) may process different numbers ofpackets from each virtual output queue of each traffic processor thatbuffers packets for the same egress port in a round robin manner basedon the weight assigned to each virtual output queue of the virtualoutput queues associated with (that buffer packets to be sent out of) agiven egress port.

For example, consider a scenario in which virtual output queuesassociated with an egress port include three virtual output queues withweights of 2, 1, and 1, respectively. In such a scenario, the switchingfabric may process two packets from the first virtual output queue, onepacket from the second virtual output queue, one packet from the thirdvirtual output queue, and then repeat the aforementioned process toprocess packets of the virtual output queues associated with the egressport.

Switching fabric (118) may perform other methods for switching packetsbased on weights assigned to each virtual output queue without departingfrom embodiments disclosed herein. For example, switching fabric (118)may employ time slicing (e.g., processing packets from a first virtualoutput queue for a predetermined amount of time based on a first weight,processing packets from a second virtual output queue for a secondpredetermined amount of time based on a second weight, etc.) or othermethods of processing packets from different virtual output queues toprocess packets from all virtual output queues associated with an egressport (and/or traffic class).

The weight assigned to each virtual output queue may be stored as partof switching system repository (110, FIG. 1.2 ). For example, switchingfabric (118) may read weights from switching system repository (110,FIG. 1.2 ) and process packets as specified by the read weights.Switching system manager (108, FIG. 1.2 ) may maintain weights stored inswitching system repository (110, FIG. 1.2 ). The aforementioned weightsmay be stored in other locations without departing from embodimentsdisclosed herein.

Switching fabric (118) may implement any protocol for distributingpackets from virtual output queues (116) to egress ports (e.g., 104A,104N) without departing from embodiments disclosed herein.

Switching fabric (118) may be implemented using one or more physicaldevices. A physical device (e.g., a chip, die, etc.) may includecircuitry adapted to perform the functionality of switching fabric(118). In some embodiments disclosed herein, the circuitry may includeprogrammable portions that may be adapted to execute computinginstructions (e.g., computer code) that cause the programmable portionsof the circuitry to perform all, or a portion, the functionality ofswitching fabric (118).

While switching system (106) of FIG. 1.3 has been illustrated asincluding a limited number of specific components, a switching systemmay include additional, fewer, and/or different components withoutdeparting from embodiments disclosed herein.

As discussed above, switching system (106) may process packets based oninformation included in a switching system repository. FIG. 1.4 shows adiagram of switching system repository (110) in accordance with one ormore embodiments disclosed herein.

Switching system repository (110) may be implemented as a data structurestored in storage. In one or more embodiments disclosed herein, thestorage is implemented using devices that provide data storage services(e.g., storing data and providing copies of previously stored data). Thedevices that provide data storage services may include hardware devicesand/or logical devices. For example, the storage may include anyquantity and/or combination of memory devices (i.e., volatile storage),long term storage devices (i.e., persistent storage), other types ofhardware devices that may provide short term and/or long term datastorage services, and/or logical storage devices (e.g., virtualpersistent storage/volatile storage).

For example, the storage may include a memory device (e.g., a dual inline memory device) in which data is stored and from which copies ofpreviously stored data are provided. In another example, the storage mayinclude a persistent storage device (e.g., a solid state disk drive) inwhich data is stored and from which copies of previously stored data isprovided. In a still further example, the storage may include a memorydevice (e.g., a dual in line memory device) in which data is stored andfrom which copies of previously stored data is provided and a persistentstorage device that stores a copy of the data stored in the memorydevice (e.g., to provide a copy of the data in the event that power lossor other issues with the memory device that may impact its ability tomaintain the copy of the data cause the memory device to lose the data).

Switching system repository (110) may include data structures includingswitching system operational parameters (132) and switching systemstates (134). Each of these data structures is discussed below.

Switching system operational parameters (132) may be included in one ormore data structures that include parameters that control the manner inwhich one or more switching systems operate. For example, switchingsystem operational parameters (132) may include any number of parametersthat indicate how switching systems are to operate. The switchingsystems may read or otherwise obtain information based on theinformation included in switching system operational parameters (132)and process packets in a manner consistent with the obtainedinformation. For additional details regarding switching systemoperational parameters (132), refer to FIG. 1.5 .

Switching system states (134) may be included in one or more datastructures that include information regarding the operational states ofone or more switching systems. For example, switching system states(134) may include information regarding (i) rates at which packets arebeing added to virtual output queues of the switching systems, (ii)rates at which packets are able to be provided to other devices viaegress ports, (iii) the quantities of storage resources that are and/ormay be allocated to each of the virtual output queues of the switchingsystems, (iv) latency states of the virtual output queues of switchingsystems, and/or (v) other types and quantities of information regardingthe operation of the switching system.

The latency states of virtual output queues may be representations ofthe time during which packets are buffered in virtual output queues.When a packet is enqueued to a virtual output queue, the packet may betemporarily stored in the virtual output queue until dequeued by aswitching fabric. Latency states of the virtual output queues mayrepresent the duration of time that the packets are temporarily storedin the respective virtual output queues.

In one or more embodiments disclosed herein, latency states arediscretized. The actual latency of a virtual output queue may be aspecific duration of time. To limit the amount of information maintainedregarding each virtual output queue, only discretized latency states maybe maintained as part of the switching system states (134). For example,each virtual output queue may have a discrete, finite number of latencystates (e.g., 10 states).

Each of the latency states may be associated with one or more parametersfor operating switching systems. Switching system manager (108, FIG. 1.2) may use the aforementioned parameters to select appropriate parameterswith which to operate switching system (106, FIG. 1.2 ).

Parameters associated with latency states may be selected in any mannerwithout departing from embodiments disclosed herein. For example, anadministrator or other person may associate the parameters with thelatency states. In other embodiments, statistical analysis of previousoperation of the network device may be used to associate the parameterswith the latency states. The parameters associated with the latencystates may be selected via other methods without departing fromembodiments disclosed herein.

Any of the data structures illustrated in FIG. 1.4 may be implementedusing structures such as, for example, lists, tables, databases, linkedlists, or any other type of structure. Any of the data structures may beimplemented using similar or different structures.

While the data structures illustrated in FIG. 1.4 have been described asincluding a limited amount of specific information, any of theaforementioned data structures may include additional, less, and/ordifferent information without departing from the embodiments disclosedherein. Further, the aforementioned data structures may be combined withother data structures, subdivided into any number of data structures,may be stored in other locations (e.g., in a storage hosted by anotherdevice), and/or span across any number devices without departing fromthe embodiments disclosed herein.

Turning to FIG. 1.5 , FIG. 1.5 shows a diagram of switching systemoperational parameters (132) in accordance with one or more embodimentsdisclosed herein. As discussed above, switching system operationalparameters (132) may be one or more data structures that includeinformation upon which a switching system bases its operation (e.g.,ordering of processing packets from virtual output queues, storageresources allocated to each of the virtual output queues, etc.).

Switching system operation parameters (132) may include any number ofentries (e.g., 140, 148). Each entry may be associated with acorresponding virtual output queue and include information regarding howthe virtual output queue is to be managed by the switching system. Forexample, each of the entries may include processing rate (142), droprate (144), and/or storage resources allocation (146).

Processing rate (142) may specify the rate at which packets stored inthe associated virtual output queue are to be processed. The rate may bespecified as, for example, an absolute or relative value. For example,if specified as a relative value (e.g., relative with respect to thepacket processing rates to be afforded to the other virtual outputqueues of a virtual output queue), the relative value may be expressedas a weight.

The weight may specify, for example, how many packets are to beprocessed when a round of packets are processed from the virtual outputqueues associated with an egress port (and/or traffic class). Differentvirtual output queues associated with an egress port may be assigneddifferent weights resulting in the packets buffered by each of thevirtual output queues to be processed at different rates.

Drop rate (144) may include information regarding the rate at whichpackets, that are to be added to the virtual output queue associatedwith the entry, are dropped prior to being enqueued. For example, droprate (144) may be specified as a percentage that specifies theprobability that any packet to be added to the virtual output queueassociated with the entry will be dropped rather than added to thevirtual output queue. Dropped may mean deleting the packet or otherwisenot continuing processing of the packet for forwarding purposes (e.g.,not adding to a virtual output queue for buffering purposes).

Storage resources allocation (146) may specify a quantity of storageresources that are to be allocated to the virtual output queueassociated with the entry. Storage resources allocation (146) mayspecify, for example, an absolute quantity of resources or a relativequantity of resources (with respect to, for example, the total quantityof resources that may be allocated to all of the virtual output queuesof a traffic processor).

The aforementioned information included in each entry may be read orotherwise obtained/used by a switching system. Once obtained, theswitching system may operate in a manner consistent with the obtainedinformation.

For example, the switching system may utilize any number of tables,stored in traffic processors (e.g., 112A, FIG. 1.3 ), to determine howto perform packet switching. The traffic processors may implement one ormore algorithms that are used to determine how packets are to beswitched when received by a network device. The algorithms used todetermine how packets are to be switched may use, in part, theinformation stored in the tables when making switching decisions. Theinformation stored in the tables may be based, in part, on the entriesin the switching system operational parameters (132). Once programmed,the switching system may operate independently and in accordance withits programming until the switching system is reprogrammed.

The switching system may be programmed using the switching systemoperational parameters (132) in any manner without departing fromembodiments disclosed herein. For example, information used to populatethe tables of the packet processors may be obtained using, in part, theswitching system parameters. The tables may then be populated by writingthe information to the tables. The switching system may be programmedusing additional, less, and/or different information from thatillustrated in FIG. 1.4 without departing from embodiments disclosedherein.

In one or more embodiments disclosed herein, a portion of the switchingsystem (or another entity distinct from the switching system such as anapplication hosted by a network device), often referred to as thecontrol plane, may program elements of the switching system (oftenreferred to as the data plane) based on any quantity of information(e.g., including, for example, the switching system parameters,information regarding the network topology in which the network deviceresides, etc.). For example, a switching system may include programmabletraffic processors that switch packets to virtual output queues based ontheir respective programming. Once programmed, the system may performpacket forwarding and switching in accordance with the programming.

While switching system operational parameters (132) are illustrated inFIG. 1.5 in a manner that includes a limited amount of specificinformation, switching system operational parameters (132) may includeaddition, less, and/or different information without departing fromembodiments disclosed herein. Further, switching system operationalparameters (132) may be combined with other data structures, subdividedinto any number of data structures, may be stored in other locations(e.g., in a storage hosted by another device), and/or spanned across anynumber devices without departing from the embodiments disclosed herein.

As discussed above, the switching system may utilize virtual outputqueues for packet switching purposes. By doing so, relationships betweenthe ingress and egress ports may be established that facilitateforwarding of packets. FIG. 1.6 shows a diagram of relationships thatmay be present in a switching system in accordance with one or moreembodiments disclosed herein.

The relationship diagram of FIG. 1.6 includes illustrations of tworelationships. The first relationship is between a port and virtualoutput queues. Specifically, as seen from the top portion of thediagram, any number of virtual output queues (152, 154) may beassociated with an egress port. In other words, any number of packetprocessors may each include a virtual output queue in which packets tobe sent out of egress port (150) are enqueued. Consequently, packetsenqueued in each of the associated virtual output queues (152, 154)corresponding to egress port (150) are systematically dequeued and sentout of egress port (150). Consequently, the rates of dequeuing packetsfrom virtual output queues (152, 154) is capped at the rate packets maybe sent of out egress port (150).

The second relationship is between virtual output queue (155) andoperational parameters (e.g., 132, FIG. 1.4 ). The operation parametersinclude ingress rate (156), processing rate (158), and drop rate (160)of the switch system which are associated with the virtual output queue(155). The operational parameters (e.g., 156, 158, 160) may reflectwhether (i) virtual output queue (155) is being filled with packets,(ii) virtual output queue (155) is maintaining a same number of packetsin virtual output queue (155), and/or (iii) virtual output queue (155)is being emptied of packets. Specifically, the second relationshipspecifies that ingress rate (156) of packets into virtual output queue(155), processing rate (158) of packets in virtual output queue (155),and drop rate (160) of packets prior to being enqueued in virtual outputqueue (155) are associated with virtual output queue (155). Theaforementioned parameters may determine whether increasing numbers,decreasing number, or stable numbers of packets are in virtual outputqueue (155). Similar relationships may exist for each virtual outputqueue of a network device.

As discussed above, switching system (e.g., 106, FIG. 1.2 ) may switchpackets (e.g., direct toward different devices) to facilitate managementof packet flow across a network. By switching packets, the packets maybe directed toward different network devices in accordance with, forexample, a packet traffic management plan. FIGS. 2.1-2.2 show diagramsof methods that may be performed by a network device in accordance withone or more embodiment disclosed herein when managing the flow ofpackets.

FIG. 2.1 shows a flowchart describing a method for processing packets inaccordance with one or more embodiments disclosed herein. The method maybe performed by, for example, a switching system manager (e.g., 108,FIG. 1.2 ) of a network device. Other entities may perform the method ofFIG. 2.1 without departing from embodiments disclosed herein.

While the various steps in the flowchart shown in FIG. 2.1 are presentedand described sequentially, one of ordinary skill in the relevant art,having the benefit of this document, will appreciate that some or all ofthe steps may be executed in different orders, that some or all of thesteps may be combined or omitted, and/or that some or all of the stepsmay be executed in parallel.

In Step 200, a change in a state of a virtual output queue of aswitching system is identified. The operational parameters associatedwith the virtual output queue may be monitored to identify the change inthe state of the virtual output queue. The operational parameters mayinclude one or more of: (i) a rate of packets being enqueued in thevirtual output queue, (ii) a latency of the virtual output queue (e.g.,duration of time between when packets and enqueued in and dequeued froma virtual output queue), and (iii) the amount of available storageresources (e.g., for storage of enqueued packets) of the virtual outputqueue. A change in one or more of the aforementioned operationalparameters may indicate a change in the state of the virtual outputqueue.

For example, consider a scenario in which two virtual output queues oftwo different traffic processors both enqueue packets to be sent out ofthe same egress port. Each of the virtual output queues are allocated 50megabytes of storage for storing (e.g., buffering) packets. Both virtualoutput queues are currently storing 20 megabytes worth of packets. Theswitching fabric processes packets from each of the virtual outputqueues at the same rate. At a point in time, the rate of packets beingadded to the first virtual output queue increases resulting in the firstvirtual output queue storing 45 megabytes worth of packets. Based on theaforementioned scenario, monitoring of the operational parametersassociated with the first virtual output queue indicates that a changein the state of the first virtual output queue has occurred due to theincreased utilization (e.g. 45 megabytes versus 20 megabytes) of thestorage resources allocated to the first virtual output queue.

In another example, consider a scenario in which two virtual outputqueues both enqueue packets to be sent out of the same egress port. Eachof the virtual output queues are allocated 50 megabytes of storage forstoring (e.g., buffering) packets. Both virtual output queues arecurrently storing 20 megabytes worth of packets. The switching fabricprocesses packets from each of the virtual output queues at the samerate. At a point in time, the rate of packets being added to the firstvirtual output queue increases resulting in the switching fabricpreferentially processing packets from the first virtual output queue toensure that the first virtual output queue does not run out of storageresources for buffering packets. Consequently, the duration of time thatpackets are buffered in the second virtual output queue greatlyincreases. Based on the aforementioned scenario, monitoring of theoperational parameters associated with the first virtual output queueindicates that a change in the state of the second virtual output queuehas occurred due to the increased latency (e.g., buffering time) ofpackets in the second virtual output queue.

In a still further example, consider a scenario in which two virtualoutput queues both enqueue packets to be sent out of the same egressport. At a point in time, the rate of packets being added to the firstvirtual output queue increases. Based on the aforementioned scenario,monitoring of the operational parameters associated with the firstvirtual output queue indicates that a change in the state of the firstvirtual output queue has occurred due to the increased rate of packetsbeing added to the first virtual output queue.

The identified change may be a change in multiple dimensions of thestate of the virtual output queue. For example, the state of the virtualoutput queue may reflect multiple operational parameters of the virtualoutput queue. When, for example, both the amount of used storageresources of the virtual output queue increases and the duration of timepackets are buffered in the virtual output queue increases, twodimensions of the state of the virtual output queue have changed.

The change in the state of the virtual output queue of the switchingsystem may be identified based on a latency state of the virtual outputqueue. For example, a network device may monitor a latency state of eachof its virtual output queues as discussed above. A change in the latencyof the virtual output queue may indicate a change in the state of thevirtual output queue.

In Step 202, an operation of the virtual output queue is modified basedon the change in the state of the virtual output queue of the virtualoutput queue.

In one or more embodiments disclosed herein, the modification includesone or more of: (i) a change in a drop rate of packets prior toenqueuing in the virtual output queue, (ii) a change in a drop rate ofpackets prior to buffering in a second virtual output queue (e.g., oneof multiple virtual output queues that all enqueue packets to be sentout of the same egress port), (iii) a change in a processing rate ofpackets buffered in the virtual output queue, (iv) a change in aprocessing rate of packets buffered in the second virtual output queue,(v) a change in an allocation of storage to the virtual output queue,and (vi) a change in an allocation of storage to a second virtual outputqueue.

In one or more embodiments disclosed herein, the drop rates of thevirtual output queue and/or the second virtual output queue are modifiedbased on the latency states of the aforementioned virtual output queues.Latency states may be respective states of the virtual output queuesthat reflect the duration of time packets are buffered in the virtualoutput queues. The latency states of virtual output queues may bemaintained as part of switching system states (134, FIG. 1.4 ) (e.g., ona per virtual output queue basis).

The network device may maintain the switching system states by (i)monitoring the latency of packets buffered in each of the virtual outputqueues, (ii) discretizing the monitored latencies into latency states,and (iii) maintaining the switching system states based on the latencystates. By doing so, information regarding the latency states of thevirtual output queues may be maintained as part of the switching systemstates.

To quantize the monitored latency into a latency state, a functionalrelationship between the latency of a virtual output queue and a latencystate may be utilized. For example, the functional relationship mayspecify the latency state as a function of the transit latency (e.g.,time between enqueuing in and dequeueing from a virtual output queue) ofthe latest packet dequeued from the virtual output queue.

The functional relationship may be a discretized form (e.g., associatesranges of monitored latencies with different latency states) of a linearfunction, exponential function, or any other type of function. Forexample, the functional relationship may associate a first latency rangewith a first latency state, a second latency range with a second latencystate, etc.

Each of the latency states may be associated with a corresponding dropprobability. The drop probability may increase with increasing latencystate until reaching a maximum drop probability up to 100% at apredetermined maximum latency. The maximum latency may be any value. Forexample, the maximum latency may be an amount of latency above whichresults in negative impacts on the ability of the network environment inwhich the network device resides to control the flow of packets withinthe network environment. The maximum latency may be above or below theaforementioned threshold without departing from embodiments disclosedherein.

Modifying the drop rates of the virtual output queue and/or the secondvirtual output queue based on the latency states of the respectivevirtual output queues may regulate the latency of packets buffered ineach of the virtual output queues. For example, increasing the drop ratewith increasing latency of packets in the virtual output queues maycause the number of packets in a virtual output queue to be maintainedbelow a predetermined number. By virtue of preventing the number ofpackets in the virtual output queue from exceeding the predeterminednumber, the corresponding latency of the packets buffered in the virtualoutput queue may be prevented from exceeding a predetermined duration.

To implement the aforementioned modifications of the drop rates, thedrop rate (e.g., 144 of FIG. 1.5 ) corresponding to each of theaforementioned virtual output queues may be added to correspondingentries of switching system operational parameters (132), as discussedwith respect to FIG. 1.5 .

In one or more embodiments disclosed herein, the processing rate of thevirtual output queue and/or the second virtual output queue are modifiedbased on (i) the rate that packets of the virtual output queue are beingprocessed (e.g., the maximum rate at which the switch fabric and egressport are able to provide packets from the virtual output queue to otherdevices) and (ii) the rates that packets are being added to each of thevirtual output queues associated with a corresponding egress port (e.g.,all of the virtual output queues that buffer packets that will be sentout of a particular egress port). For example, the rate at which thepackets of the virtual output queues associated with an egress port arebeing processed may be the maximum rate at which the switching fabric isable to process packets of these virtual output queues (referred to asthe virtual output queue processing rate).

The rates at which packets of each of the virtual output queuesassociated with an egress port are processed may be determined based onthe virtual output queue processing rate and the rates that packets arebeing added to each of the virtual output queues associated with theegress port. In other words, the processing rate of the virtual outputqueue may be allocated across the virtual output queues associated withthe egress port. The processing rate of the virtual output queue may beproportionally allocated to the virtual output queues based on therelative rates at which packets are being added to the virtual outputqueues associated with the egress port.

For example, consider a scenario in which packets are being processed at100 gigabits per second from the virtual output queues associated withan egress port, packets are being added at a rate of 150 gigabits persecond to a first virtual output queue of the virtual output queuesassociated with the egress port, and packets are being added at a rateof 50 gigabits per second to a second virtual output queue of thevirtual output queues associated with the egress port. In such ascenario, packets from the first virtual output queue are processed at arate of 75 gigabytes per second (150/(150+50)×100) and packets from thesecond virtual output queue are processed at a rate of 25 gigabits persecond (50/(150+50)×100) (e.g., a proportional allocation of the rate atwhich packets can be dequeued from all of the virtual output queuesassociated with the egress port).

To allocate the virtual output queue processing rate across virtualoutput queues associated with an egress port, relative weights (e.g.,processing rate (142) of FIG. 1.5 ) corresponding to each of theaforementioned virtual output queues may be added to correspondingentries of switching system operational parameters (132), as discussedwith respect to FIG. 1.5 . For example, processing weights (e.g., 142)of 0.75 and 0.25 could be added to corresponding entries of switchingsystem operational parameters (132, FIG. 1.5 ). Consequently, theswitching fabric may process packets in an order that causes packetsfrom the first and second queues to be processed at the rates of 75 and25 gigabits per second, respectively.

Modifying the processing rates of packets of the virtual output queueand/or second virtual output queue may change the manner in whichpackets that would be queued in the virtual output queue and/or thesecond virtual output queue are treated by a switching system. It may bemore or less likely that the aforementioned packets are dropped prior tobeing buffered (e.g., prior to being enqueued) in the virtual outputqueues. Consequently, the latency of each of the virtual output queuesassociated with an egress port may be maintained by virtue of thechanged drop likelihood (e.g., drop rate).

In one or more embodiments disclosed herein, the allocation of storageto the virtual output queue and/or the second virtual output queue aremodified based on (i) the rates that packets are being added to each ofthe virtual output queues of the virtual output queue and (ii) a targetlatency of packets in the virtual output queues.

When packets are added to a virtual output queue at a rate that isgreater than a rate at which the packets are processed (e.g., dequeued),the number of buffered packets may increase. When more packets arebuffered in a virtual output queue, the time each packet is buffered mayincrease. Consequently, the latency of packets buffered in the virtualoutput queue may increase. Increased latencies may have negativeconsequences for packet flow control in a network. For example, if thelatency of a packet is too large, components of a network may (i)consider the packet to be lost and/or (ii) request that an additionalpacket that includes information similar to that included in thebuffered but considered lost packet be sent. Doing so may result inincreased packet transmissions (e.g., network traffic) and/or discardingof the packet by components of a network after being buffered in avirtual output queue.

Embodiments disclosed herein may provide a method for limiting packetlatency in virtual output queues by allocating storage to the virtualoutput queues associated with an egress port based on (i) the rates thatpackets are being added to the virtual output queues and (ii) the ratesat which packets of the virtual output queues are being processed. Thestorage allocations may be made to ensure that no packets are stored inthe virtual output queues beyond the target latency. By doing so, theactual latencies of packets buffered in the virtual output queue may notexceed the target latency.

For example, consider a scenario where packets of a virtual output queueare being processed at a rate of 20 gigabits per second and the virtualoutput queue has a target latency of 1 millisecond. In this scenario,2.5 megabytes (20 gigabits per second×1 millisecond) are allocated tothe virtual output queue for buffering purposes.

To allocate the storage resources to the virtual output queues, storageresource allocations (e.g., 146 of FIG. 1.5 ) corresponding to each ofthe aforementioned virtual output queues associated with an egress portmay be added to corresponding entries of switching system operationalparameters (132), as discussed with respect to FIG. 1.5 .

In one or more embodiments disclosed herein, the drop rates, processingrates, and/or allocations of storage of the virtual output queuesassociated with an egress port are managed independently. For example,the aforementioned rates and/or allocations may change independently ofone another. The changes in rates and/or allocations may be separatelymodified to prevent the latency of packets buffered in virtual outputqueues from exceeding predetermined amounts of time. The predeterminedamounts of time may be determined based on, for example, packet routinggoals of a network in which a network device resides or for otherpurposes. The predetermined amounts of time may be based on otherfactors without departing from embodiments disclosed herein.

Modifying the allocations of storage to the virtual output queue and/orsecond virtual output queue may change the manner in which packets thatwould be queued in the virtual output queue and/or the second virtualoutput queue are treated by a switching system. Allocating storage basedon the rates packets are being removed from virtual output queues maylimit the latencies of the virtual output queues. Allocating storageproportionally to virtual output queues based on the rates at whichpackets being removed from the virtual output queues may cause themaximum durations of time packets are allowed to be buffered in each ofthe respective virtual output queues to be matched to each other.

In step 204, a packet associated with the virtual output queue isobtained. The packet may be obtained via, for example, an ingress port.The packet may be provided by another device operably connected to anetwork device via the ingress port. The packet be addressed to anotherdevice (e.g., not the network device that obtained the packet).

In step 206, it is determined whether a drop rate indicates that thepacket is to be dropped. The drop rate may be associated with thevirtual output queue. The drop rate may be stored as part of switchingsystem operational parameters, as discussed with respect to FIG. 1.5 .

The determination may be made based on the probability indicated by thedrop rate. The probability may be used in conjunction with, for example,a random number generator or other function to ascertain whether thepacket is to be dropped. For example, the probability may specify arange of numbers, that the random number generator may generate, thatindicates that the packet is to be dropped. The random number generatormay generate a random number. If the random number falls within therange of numbers, the drop rate indicates that the packet is to bedropped. The determination may be made in different manners withoutdeparting from embodiments disclosed herein.

In one or more embodiments disclosed herein, the drop rate is specifiedby a functional relationship. The functionality relationship may specifya probability of dropping the packet as a function of the latency of thequeue into which the packet will be enqueued.

The functional relationship may specify that the packet will not bedropped while the queue into which the packet will be enqueued has alatency below a first predetermined latency. In other words, a zeropercent chance (e.g., 0% drop probability) of being dropped. The firstpredetermined latency may be referred to as a minimum latency.

The functional relationship may also specify that the packet will belikely to be dropped prior to being enqueued while the queue into whichthe packet will be enqueued has a latency above a second predeterminedlatency. In other words, the packet may have a probability of beingdropped prior to be enqueued of a maximum, predetermined rate. Themaximum predetermined rate may be 100% or less than 100% (e.g., 90%,80%, etc.). The second predetermined latency may be referred to as amaximum latency.

The functional relationship may further specify that the packet may bedropped prior to being enqueued while the queue into which the packetwill be enqueued has a latency greater than the first predeterminedlatency but less than the second predetermined latency. The drop ratemay be between 0% and the maximum, predetermined rate (e.g., between 0%and 100% if the maximum, predetermined rate is 100%; between 0% and 90%if the maximum, predetermined rate is 90%; etc.) The drop rate may bespecified using a continuous or quantized function.

For example, a linear, exponential, or other type of function may beused to specify the probability that a packet will be dropped when thequeue into which the packet will be enqueued has a latency between thefirst predetermined latency and second predetermined latency. Thefunction may specify a 0% drop probability when the queue into which thepacket will be enqueued has a minimum latency and a maximum,predetermined value when the queue into which the packet will beenqueued has a maximum latency. If the drop probability is implementedusing a linear function, then the drop probability may rangecontinuously between 0% and the maximum, predetermined value over therange of minimum latency to maximum latency.

In some embodiments disclosed herein, the latency of a queue (e.g., avirtual output queue) may be quantized. In other words, the latency of aqueue may be categorized based on discrete ranges. Consequently, whenthe latency of a queue is obtained, it may only take on one of a few,discrete values. The corresponding drop probability may also correspondto only a few, discretized value by virtue of the relationship betweenthe discretized latencies of the queues and the drop probabilities.

By utilizing quantized latency states and corresponding drop rates, thetotal amount of storage required to store latency states and determinedrop rates may be reduced when compared to utilized non-quantized and/ordiscretized representations. For additional details regarding utilizingquantized latency states and determining corresponding drop rates, referto FIG. 3.6 .

If the drop rate indicates that the packet is to be dropped, the methodmay proceed to step 208. If the drop rate does not indicate that thepacket is to be dropped, the method may proceed to step 210.

In step 208, the packet is discarded. In other words, the packet isdropped and not buffered in the virtual output queue.

The method may end following step 208.

Returning to step 206, the method may proceed to step 210 if the droprate does not indicate that the packet is to be dropped.

In step 210, the packet is added to the virtual output queue.

The method may end following step 210.

Using the method illustrated in FIG. 2.1 , a network device may processpackets in a manner that limits the latency of the packets in virtualoutput queues. By doing so, negative downstream impacts of high latencypackets may be avoided. Consequently, routing of packets within anetwork environment in which the network device resides may be improved.

As noted above, the network device may modify the manner in whichpackets are processed by a switching system to limit the latency of thepackets. To do so, the network device may monitor the operationalparameters of the switching system and update the latency state of theswitching system.

FIG. 2.2 shows a flowchart describing a method for maintaining latencystates in accordance with one or more embodiments disclosed herein. Themethod may be performed by, for example, a switching system manager(e.g., 108, FIG. 1.2 ). Other entities may perform the method of FIG.2.2 without departing from embodiments disclosed herein.

While the various steps in the flowchart shown in FIG. 2.2 are presentedand described sequentially, one of ordinary skill in the relevant art,having the benefit of this document, will appreciate that some or all ofthe steps may be executed in different orders, that some or all of thesteps may be combined or omitted, and/or that some or all of the stepsmay be executed in parallel.

In step 210, a change in latency of packets buffered by a virtual outputqueue of virtual output queues associated with an egress port (e.g., allof which that enqueue packets to be sent out of the egress port) of aswitching system is identified.

The change in latency may be identified by monitoring the latency ofpackets buffered by the virtual output queue. The latency may bemonitored by ascertaining when packets are dequeued from the virtualoutput queue. The latency may be the difference in time between when thepackets are enqueued to and dequeued from the virtual output queue.

The monitoring may be performed by, for example, sampling the latency ofpackets dequeued from the virtual output queue. The sampling may beperformed at one or more points in time.

Sampling the latency of packets may include statistically analyzing thelatency of multiple packets dequeued from the virtual output queue. Thestatistical analysis may include averaging to obtain an average latencyassociated with a point in time and/or a period of time.

When the latency of the packets dequeued from the virtual output queuechanges by a predetermined amount, the change in latency is identified.The predetermined amount may be any amount.

The predetermined amount may correspond to a change in latency range. Asdiscussed above, the latency states of a virtual output queue maycorrespond to different latency ranges. The predetermined amount may beany amount that causes the latency of the virtual output queue to changelatency ranges.

For example, consider a scenario in which a first latency state isassociated with a first latency range of less than 0.5 milliseconds anda second latency state is associated with a second latency range ofgreater than 0.5 milliseconds. If the latency of the virtual outputqueue is 0.4 milliseconds then the predetermined amount may be anincrease of 0.1 milliseconds.

In step 212, a representation of a latency state of the virtual outputqueue is modified based on the change in latency. A portion of switchingsystem states (e.g., 134, FIG. 1.4 ) corresponding to the latency of thevirtual output queue may be modified to modify the representation of thelatency state of the virtual output queue.

The representation of the latency may be, for example, a discretizedrepresentation of the latency (i.e., one of a finite number ofrecognized latency states). The representation of the latency may bemodified by using a functional relationship between the actual latencyand the discretized latency to identify a current latency state of thevirtual output queue. The representation of the latency may be modifiedto reflect the current latency state of the virtual output queue.

The method may end following step 212.

Using the method illustrated in FIG. 2.2 , a network device may maintainthe latency state of virtual output queues in a computationallyefficient manner. For example, the actual latency of a virtual outputqueue may be used to ascertain a latency state that may be representedin a computationally efficient manner. By doing so, the computationalcost for maintaining the latency state of virtual output queues may bereduced when compared to the computational cost for maintaining theactual latency of the virtual output queue.

In one or more embodiments disclosed herein, a network device is aphysical device that includes and/or is operatively connected topersistent storage (not shown), memory (e.g., random access memory(RAM)) (not shown), one or more processor(s) (e.g., integrated circuits)(not shown), and at least one physical network interface (not shown),which may also be referred to as a port. Examples of a network device(e.g., 10, FIG. 1.1 ) include, but are not limited to, a network switch,a router, a multilayer switch, a fiber channel device, an InfiniBand®device, etc. A network device (e.g., 10, FIG. 1.1 ) is not limited tothe aforementioned specific examples.

In one or more embodiments disclosed herein, a network device (e.g., 10,FIG. 1.1 ) includes functionality to receive packets (e.g., frames,packets, tunneling protocol frames, etc.) at any of the physical networkinterfaces (i.e., ports) of the network device (e.g., 10, FIG. 1.1 ) andto process the packets. In one or more embodiments, processing a packetincludes, but is not limited to, a series of one or more table lookups(e.g., longest prefix match (LPM) lookups, forwarding informationlookups, etc.) and corresponding actions (e.g., forward from a certainegress port, add a labeling protocol header, rewrite a destinationaddress, encapsulate, etc.). Such a series of lookups and correspondingactions may be referred to as a pipeline, and may, for example, beprogrammed as a match-action pipeline using the P4 programming language.Examples of pipeline processing include, but are not limited to,performing a lookup to determine: (i) whether to take a security action(e.g., drop the network traffic data unit); (ii) whether to mirror thepacket traffic on the network; and/or (iii) determine how toroute/forward the packet in order to transmit the packet from aninterface of the network device (e.g., 10, FIG. 1.1 ). Switching system(106, FIG. 2 ) may perform all, or a portion, of the pipelineprocessing.

In one or more embodiments disclosed herein, a network device (e.g., 10,FIG. 1.1 ) also includes and/or is operatively connected to devicestorage and/or device memory (i.e., non-transitory computer readablemediums used to provide persistent storage resources and/or memoryresources) storing software and/or firmware.

Such software and/or firmware may include instructions which, whenexecuted by the one or more processors) of a network device (e.g., 10,FIG. 1.1 ), cause the one or more processors to perform operations inaccordance with one or more embodiments described herein.

The software instructions may be in the form of computer readableprogram code to perform embodiments described herein, and may be stored,in whole or in part, temporarily or permanently, on a non-transitorycomputer readable medium such as a CD, DVD, storage device, a diskette,a tape, flash memory, physical memory, or any other non-transitorycomputer readable medium.

In one or more embodiments, a network device (e.g., 10, FIG. 1.1 ) ispart of a network (e.g., 20, FIG. 1.1 ). A network may refer to anentire network or any portion thereof (e.g., a logical portion of thedevices within a topology of devices). A network may include adatacenter network, a wide area network, a local area network, awireless network, a cellular phone network, or any other suitablenetwork that facilitates the exchange of information from one part ofthe network to another. In one or more embodiments, the network may becoupled with or overlap, at least in part, with the Internet.

In one or more embodiments, a network includes a collection of one ormore network devices (e.g., network device (e.g., 10, FIG. 1.1 )) thatfacilitate network connectivity for one or more operatively connecteddevices (e.g., computing devices, data storage devices, other networkdevices, etc.). In one or more embodiments, a network device (e.g., 10,FIG. 1.1 ) and other devices (e.g., 5, 30, FIG. 1.1 ) within a networkare arranged in a network topology. In one or more embodiments, anetwork topology is an arrangement of various elements of a network.

In one or more embodiments, a computing device (e.g., 400, FIG. 4 ) isany device or any set of devices capable of electronically processinginstructions and may include, but is not limited to, any of thefollowing: one or more processors (not shown), memory (e.g., randomaccess memory (RAM)) (not shown), input and output device(s) (notshown), persistent storage (not shown), one or more physical interfaces(e.g., network ports) (not shown), any number of other hardwarecomponents (not shown) or any combination thereof. Examples of computingdevices include, but are not limited to, a server (e.g., a blade-serverin a blade-server chassis, a rack server in a rack, etc.), a desktopcomputer, a mobile device (e.g., laptop computer, smart phone, personaldigital assistant, tablet computer and/or any other mobile computingdevice), a network device (e.g., switch, router, multi-layer switch,etc.) such as that described above and below, a virtual machine, and/orany other type of computing device with the aforementioned requirements.

To further clarify embodiments disclosed herein, a non-limiting exampleis provided in FIGS. 3.1-3.6 . FIGS. 3.1-3.5 may illustrate a system, orportions thereof, similar to that illustrated in FIG. 1.1 at differentpoints in time. FIG. 3.6 may illustrate a relationship used by thesystem of FIG. 3.1 . For the sake of brevity, only a limited number ofcomponents of the system of FIG. 1.1 is illustrated in each of FIGS.3.1-3.5 .

Example

Consider a scenario as illustrated in FIG. 3.1 in which a network deviceis providing packet forwarding services as part of a network. To providethe forwarding services, the network device includes a switching systemthat switches packets between ingress and egress ports. The switchingsystem includes three traffic processors (300, 302, 304). In FIGS.3.1-3.5 , packets are illustrated as flowing from left to right in thefigures while being processed by the switching system.

To facilitate switching of packets, first traffic processor (300)includes a first virtual output queue (not shown) into which packets arebeing deposited at a rate of 200 gigabits per second (Gbps). The secondtraffic processor (302) includes a second virtual output queue (notshown) into which packets are being deposited at a rate of 55 Gbps. Bothof these virtual output queues buffer packets that will be provided toother devices via egress port (310). Egress port (310) is able toprovide packets to other devices at a rate of 100 Gbps. Consequently,packets can only be dequeued from these virtual output queues at a rateof 100 Gbps

At the point in time illustrated in FIG. 3.1 , the switching fabric (notshown) of the switching system is processing packets from the firstvirtual output queue and the second virtual output queue at 50 Gbpseach, which saturates egress port (310). Additionally, while notillustrated in FIG. 3.1 , 55 Megabytes of storage are allocated to eachof the virtual output queues.

Based on the above description, packets are accumulating in the firstvirtual output queue of first traffic processor (300) because packetsare being enqueued to the first virtual output queue at a rate of 200Gbps while packets are being dequeued by virtue of processing by theswitching fabric at a rate of 50 Gbps. Packets are also accumulating inthe second virtual output queue of second traffic processor (302)because packets are being enqueued faster than they are being dequeued.

Based on the change in the number of packets in the first virtual outputqueue (i.e., the accumulation), the switching system manager (not shown)of the network device identifies that a change in the state of theswitching system (more specifically the virtual output queue of firsttraffic processor (300)) has occurred and modifies the operationalparameters of the switching system as shown in FIG. 3.2 .

As seen in FIG. 3.2 , the switching system manager modified theoperational parameters of the switching system to increase the rate atwhich packets are being processed from the first virtual output queue to78.4 Gbps and the rate at which packets are being processed from thesecond virtual output queue to 21.5 Gbps. By doing so, the maximum rateat which packets can be dequeued from the virtual output queues (i.e.,100 Gbps) has been proportionally divided based on the rates at whichthe packets are being added to the virtual output queues (e.g.,100×(200/(200+55)) and 100×(55/(200+55))).

After modifying the rates at which packets are being processed from thevirtual output queues, packets on the first and second virtual outputqueue experience different latency because the 55 MB of buffer of eachvirtual output queue is serviced at different rates (e.g., packets arebeing dequeued at 78.4 and 21.5 Gbps, respectively). For example,packets on the second virtual output queue (e.g., of second trafficprocessor (302)) are only being dequeued at a rate of 21.5 Gbps whilepackets are being dequeued at a rate of 78.4 Gbps from the first virtualoutput queue of first traffic processor (300). Consequently, theduration of time packets are buffered in the second virtual output queuebegins to increase substantially until the packets are buffered in thesecond virtual output queue for much longer than the packets that arebuffered in the first virtual output queue.

However, based on the change in the latency of packets in the secondvirtual output queue, the switching system manager of the network deviceidentifies that a second change in the state of the switching system hasoccurred and modifies the operational parameters of the switching systemas shown in FIG. 3.3 .

As seen in FIG. 3.3 , the switching system manager modified theoperational parameters of the switching system to decrease the amount ofstorage allocated to the first virtual output queue to 39 Megabytes andthe second virtual output queue to 10.8 Megabytes. Consequently, bothvirtual output queues fill up more quickly than if left at an allocationof 50 Megabytes and begin dropping packets at a higher rate when thelatency of the packets being dequeued rises above a 4 ms target,resulting in both virtual output queues only having a latency slightlyabove 4 milliseconds. In other words, by decreasing the amount ofstorage allocated to these virtual output queues, packets are droppedprior to being enqueued in both virtual output queues resulting in bothvirtual output queues being filled but only having around 39 and 10.8Megabytes worth of data, respectively. Consequently, the processingrates of packets from the virtual output queues (i.e., 78.4 Gbps and21.5 Gbps) are able to limit the latency of the packets to 4milliseconds.

By limiting the latency of the packets to 4 milliseconds, negativedownstream impacts on the ability of the network to route packets areavoided while also signaling (e.g., indirectly by high packet drop ratesby the network device as seen by upstream entities) to upstream entitiesthat the network device is unable to process packets at the ratespackets are being directed toward the network device.

Now, consider a second scenario in which the network device is returnedto its state as illustrated in FIG. 3.2 . As noted above, the networkdevice has modified the rate at which packets are being processed byallocating its ability to process packets from the virtual output queuesassociated with egress port (310) proportionally based on the rates atwhich packets are being added to the virtual output queues.Specifically, packets are being processed at a rate of 78.4 Gbps and21.5 Gbps from the first and second virtual output queues, respectively.

However, because packets are being added to the virtual output queues ata rate greater than packets are being removed from the virtual outputqueues, the latency of the packets in both virtual output queues ischanging. Accordingly, the switch system manager identifies that a statechange in the latency state of the virtual output queues has occurred asillustrated in FIG. 3.5 . Specifically, the switch system manageridentifies that both virtual output queues have entered a high latencystate necessitating an increase in the drop rate. In steady state 61% ofthe packets would be dropped (55 Gbps−21.5 Gbps)/55 Gbps prior to beingenqueued

To make the determination that a 61% drop rate is necessitated, theswitch system manager identifies the drop rate is associated with a highlatency state. The functional relationship upon which the determinationis based is illustrated in FIG. 3.6 . FIG. 3.6 shows a plot of thefunctional relationship between the actual latency rate, the latencystates, and the drop rates corresponding to each of the latency states.

The actual latency rate is plotted along the horizontal axis and thedrop rate is plotted along the vertical axis. The functionalrelationship between the actual latency rate and the latency state isdiscretized into low, medium, and high latency states. As seen in FIG.3.6 , the discretization breaks what would be an exponential curve intothree latency states. The latency states are associated with corresponddrop rates of 5%, 30%, and 80%, as indicated by the horizontal bars. Inthis case, it is expected that the virtual output queue would oscillatebetween the 30% and 80% drop rates to achieve an effective drop rate of61%.

In this second scenario, the drop rate for both virtual output queuesgoes up to 80% because both virtual output queues have entered a highlatency state. Consequently, 80% of packets that would be added to bothvirtual output queues are dropped prior to being enqueued to the virtualoutput queues. By doing so, the rate that packets are being added to thevirtual output queues is reduced by 80% to 15.6 Gbps for the firstvirtual output queue and 4.3 Gbps for the second virtual output queue.Accordingly, the latency for both virtual output queues begins todecrease resulting in the latency of the virtual output queues having aprogressively reduced impact on the network. Once the latency of thevirtual output queues is reduced to a medium latency, the drop ratebecomes 30% and packets begin to accumulate in the virtual output queuesuntil a high latency state with a drop rate of 80% occurs. In steadystate the latency of the virtual output queues may oscillate betweenmedium and high latency states. Consequently, the latency of the packetsis bound within a desired range.

End of Example

As discussed above, embodiments disclosed herein may be implementedusing computing devices. FIG. 4 shows a diagram of a computing device inaccordance with one or more embodiments disclosed herein. Computingdevice (400) may include one or more computer processors (402),non-persistent storage (404) (e.g., volatile memory, such as randomaccess memory (RAM), cache memory), persistent storage (406) (e.g., ahard disk, an optical drive such as a compact disk (CD) drive or digitalversatile disk (DVD) drive, a flash memory, etc.), communicationinterface (412) (e.g., Bluetooth interface, infrared interface, networkinterface, optical interface, etc.), input devices (410), output devices(408), and numerous other elements (not shown) and functionalities. Eachof these components is described below.

In one embodiment disclosed herein, computer processor(s) (402) may bean integrated circuit for processing instructions. For example, computerprocessor(s) (402) may be one or more cores or micro-cores of aprocessor. Computing device (400) may also include one or more inputdevices (410), such as a touchscreen, keyboard, mouse, microphone,touchpad, electronic pen, or any other type of input device. Further,communication interface (412) may include an integrated circuit forconnecting computing device (400) to a network (not shown) (e.g., alocal area network (LAN), a wide area network (WAN) such as theInternet, mobile network, or any other type of network) and/or toanother device, such as another computing device.

In one embodiment disclosed herein, computing device (400) may includeone or more output devices (408), such as a screen (e.g., a liquidcrystal display (LCD), a plasma display, touchscreen, cathode ray tube(CRT) monitor, projector, or other display device), a printer, externalstorage, or any other output device. One or more of the output devicesmay be the same or different from the input device(s). The input andoutput device(s) may be locally or remotely connected to computerprocessor(s) (402), non-persistent storage (404), and persistent storage(406). Many different types of computing devices exist, and theaforementioned input and output device(s) may take other forms.

Embodiments disclosed herein may provide a network device and/or methodthat provide packets switching services that takes into account thelatency of packets during switching. For example, embodiments disclosedherein may provide a network device that proactively controls howpackets are being switched based on the latency states of virtual outputqueues used during switching of packets. By doing so, a network devicemay switch packets within predetermined latency goals. By doing so,downstream impacts of high latency packets may be avoided. Consequently,the computational cost for switching packets may be reduced by avoiding,for example, resending of packets or discarding of packets due to thelatency of the packets exceeding latency requirements of the network.

Thus, embodiments disclosed herein may address the problem ofdynamically changing network conditions by efficiently marshallinglimited network switching resources. For example, embodiments disclosedherein may proactively discard packets prior to buffering (e.g.,dropping prior to enqueuing in a virtual output queue) in virtual outputqueues to limit the latency of packets in virtual output queues. Bydoing so, consumption of computing resources for buffering of packetsthat will be discarded downstream due to latency concerns may beavoided.

While embodiments have been described as addressing one or more specificchallenges relating to network environments, embodiments disclosedherein are broadly applicable to addressing many networking challengesand the embodiments should not be construed as only addressing or beingusable to solve the specific challenges discussed above.

While embodiments described herein have been described with respect to alimited number of embodiments, those skilled in the art, having benefitof this disclosure, will appreciate that other embodiments can bedevised which do not depart from the scope of embodiments disclosedherein. Accordingly, the scope embodiments described herein should belimited only by the attached claims.

What is claimed is:
 1. A network device, comprising: a switching systemfor directing packets between ingress ports and egress ports of thenetwork device; and a switching system manager programmed to: make anidentification of a state change of a virtual output queue of theswitching system; and perform an action set, in response to theidentification of the state change, to oscillate a latency of thevirtual output queue between a first predetermined latency and a secondpredetermined latency lower than the first predetermined latency suchthat the latency of the virtual output queue is bound within apredetermined range based on a quality of forwarding services to beprovided by the network device.
 2. The network device of claim 1,wherein the action set comprises: modifying a drop rate for the virtualoutput queue based on the change in the state of the virtual outputqueue.
 3. The network device of claim 2, wherein the drop rate ismodified by changing a probability of dropping the packets prior toqueueing in the virtual output queue.
 4. The network device of claim 1,wherein the switching system manager is further programmed to: prior toidentifying the state change: identify a change in an ingress rate ofthe virtual output queue; in response to identifying the change in theingress rate of the virtual output queue: perform a second action set,based on the change in the ingress rate of the virtual output queue, tomodify the latency of the virtual output queue to meet the predeterminedlatency.
 5. The network device of claim 4, wherein performing the secondaction set comprises: modifying a processing rate associated with thevirtual output queue, and modifying a second processing rate associatedwith a second virtual output queue of the virtual output queue.
 6. Thenetwork device of claim 5, wherein the processing rate and the secondprocessing rate are modified based on: an egress rate of an egress portof the egress ports that is associated with the virtual output queue,the change in the ingress rate, and an ingress rate of the secondvirtual output queue.
 7. The network device of claim 1, wherein thestate change of the virtual output queue is based on a change in latencyof the virtual output queue.
 8. A method, comprising: making anidentification of a state change of a virtual output queue of aswitching system for directing packets between ingress ports and egressports of a network device; and perform an action set, in response to theidentification of the state change, to oscillate a latency of thevirtual output queue between a first latency and a second latency lowerthan the first latency such that the latency of the virtual output queueis bound within a range based on a quality of forwarding services to beprovided by the network device.
 9. The method of claim 8, wherein theaction set comprises: modifying a drop rate for the virtual output queuebased on the change in the state of the virtual output queue.
 10. Themethod of claim 9, wherein the drop rate is modified to change aprobability of dropping the packets prior to queueing in the virtualoutput queue.
 11. The method of claim 8, further comprising: prior toidentifying the state change: identifying a change in an ingress rate ofthe virtual output queue; in response to identifying the change in theingress rate of the virtual output queue: performing a second actionset, based on the change in the ingress rate of the virtual outputqueue, to modify the latency of the virtual output queue to meet thepredetermined latency.
 12. The method of claim 11, wherein performingthe second action set comprises: modifying a processing rate associatedwith the virtual output queue, and modifying a second processing rateassociated with a second virtual output queue of the virtual outputqueue.
 13. The method of claim 12, wherein the processing rate and thesecond processing rate are modified based on: an egress rate of anegress port of the egress ports that is associated with the virtualoutput queue, the change in the ingress rate, and an ingress rate of thesecond virtual output queue.
 14. The method of claim 10, wherein thestate change of the virtual output queue is based on a change in latencyof the virtual output queue.
 15. A non-transitory computer readablemedium comprising computer readable program code, which when executed bya computer processor enables the computer processor to perform a method,the method comprising: making an identification of a state change of avirtual output queue of a switching system for directing packets betweeningress ports and egress ports of a network device; and perform anaction set, in response to the identification of the state change, tooscillate a latency of the virtual output queue between a firstpredetermined latency and a second predetermined latency lower than thefirst predetermined latency such that the latency of the virtual outputqueue is bound within a predetermined range based on a quality offorwarding services to be provided by the network device.
 16. Thenon-transitory computer readable medium of claim 15, wherein the actionset comprises: modifying a drop rate for the virtual output queue basedon the change in the state of the virtual output queue.
 17. Thenon-transitory computer readable medium of claim 16, wherein the droprate is modified to adjust a probability of dropping the packets priorto queueing in the virtual output queue.
 18. The non-transitory computerreadable medium of claim 15, wherein the method further comprises: priorto identifying the state change: identifying a change in an ingress rateof the virtual output queue; in response to identifying the change inthe ingress rate of the virtual output queue: performing a second actionset, based on the change in the ingress rate of the virtual outputqueue, to modify the latency of the virtual output queue to meet thepredetermined latency.
 19. The non-transitory computer readable mediumof claim 18, wherein the method further comprises: wherein performingthe second action set comprises: modifying a processing rate associatedwith the virtual output queue, and modifying a second processing rateassociated with a second virtual output queue of the virtual outputqueue.
 20. The non-transitory computer readable medium of claim 19,wherein the processing rate and the second processing rate are modifiedbased on: an egress rate of an egress port of the egress ports that isassociated with the virtual output queue, the change in the ingressrate, and an ingress rate of the second virtual output queue.